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(54) Title: METHOD OF INSPECTING ARTICLES 



(57) Abstract 

A method of inspecting an article (1) with re* 
spect to a stored list of reference features (6) by scan- 
ning the article to detect online features (5), transform- 
ing their coordinates according to registration trans- 
formation parameters to correct for misregistration of 
the article, and comparing (7) the transformed coordi- 
nates of the online features with those of the stored 
reference features for matches. According to this 
method, an initial estimate of the required registration 
transformation parameters is provided and is dynami- 
cally improved by continuously computing and peri- 
odically updating the estimate from the coordinates of 
the online features detected during the scanning of the 
article (1) and the coordinates of the stored reference 
features (6). 
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METHOD OF INSPECTING ARTICLES 



FIELD AND BACKGROUND OF THE INVENTION 
The present invention relates to a method of 



inspecting articles for defects and the like. The invention 
is particularly useful in automatic optical inspection 
systems for printed circuit boards (PCBs), and is theirefore 
described below with respect to this application. 



inspected with respect to a reference, namely a flawless or 
10 almost flawless board. Such a reference is characterized by 
a list of features created either by processing a boaird 
known to be substantially flawless ^ or by processing 
computer aided manufacturing (CAM) data of such a board. 
The list of features may include any number of features, 
15 typically from a few thousand to tens of thousands. Each 
feature is identified by its coordinates (x,y), and by a 
code indicating the feature type and enabling the indexing 
of as many different types of features as necessary for any 
given PCB. 

20 To find defects in a PCB that is to be inspected, 

a list of features characterizing the inspected PCB 
(commonly called an online list) is prepared and is then 
compared with a reference list of the features previously 
prepared and stored for comparison purposes. In order to 

25 produce the online list, the PCB is optically scanned^ and 
its image is processed to detect the features therein. Any 
feature appearing in the online list with no match in the 
reference list is termed an "excess feature" and is reported 
as a defect. similarly, any feature appearing in the 
30 reference list with no match in the online list is termed a 
"missing feature" and is also reported as a defect. 

When the online list and the reference list are in 
perfect geometrical registration, the matching is quite 
simple since it is only necessary to compare the two lists 
35 and to determine whether the feature type and coordinates of 



Printed circuit boards are often optically 
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one match those of the other. In reality, the coordinates 
are never in perfect geometrical registation so that such a 
simple comparison procedure cannot be performed. Therefore, 
the coordinates are not required to be exactly the same, but 
5 they should be close enough, as specified by the user, to 

allow a discrepancy parameter, termed "reference tolerance", 
to be used in the matching. The reference tolerance is 
typically considerably smaller than the design grid of the 
PCB, If it is smaller than the manufacturing process 
10 precision or the feature-detector precision of the optical 
inspection system, the false alarm rate of the inspection 
will increase - 

It will thus be appreciated that the comparison of 
the inspected article (sample) with the reference is greatly 
15 facilitated by having the sample placed on the scanner with 
as perfect registration as possible to the reference. Also 
important is the geometrical accuracy of the inspection 
system itself. 

Various mechanical, optical or electronic aids are 
20 presently used for registering PCBs in order to permit their 
optical inspection with respect to a reference list of 
features , 

Mechanical aids usually take the form of 
registration pins, sideboards, and the like; see for example 

25 US Patents 4,783,825 and 4,799,175. Using such mechanical 
means requires special and different preparations for 
different types of PCBs. For example, using registration 
pins requires constructing the pins on the scanner table, 
punching holes in the articles to be inspected, and manual 

30 or other mechanical alignment of the holes with the 
registration pins. Such mechanical aids thus involve 
significant additional costs and time for this extra 
mechanical handling. 

Computational registration is often achieved by 

35 having easily detected fiducial marks in perfect 

registration with the PCB pattern; see for example European 
Patent Application 0206712, This method requires an extra 
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scan for the fiducials prior to inspection, and therefore 
significantly increases the overall inspection time. Only 
after the fiducials have been. scanned can the 

misregistration be calculated and fixed. Also, the fiducial 

5 marks require allocations of extra space on the PCBs in 
addition to the functional PCB pattern. 

Some systems, for example as disclosed in US 
Patent 4,805,123, carry out local registration by pixel-wise 
correlating reference image to online image. This requires 

10 the reference images to be stored, instead of just the 
characteristic features. Because of the extreme 
computational complexity of carrying out correlations 
between images, such a system can only compensate for 
smaller amounts of misregistration and can be implemented, 

15 as a practical matter, only on special-purpose hardware. 

Still other systems, such as described in European 
Patent Application 0247308, employ dynamic distortion 
control to compensate for local distortions, rather than for 
global misregistration. Such systems, therefore, still 

20 require preliminary registration, for example according to 
one of the above-described techniques. 



OBJECTS AND BRIEF SUMMARY OF THE INVENTION 
An object of the present invention is to provide a 
method having advantages in the above respects for 
25 inspecting an article with respect to a stored list of 
reference features. 

More particularly, an object of the present 
invention is to provide a method of inspecting an article 
with respect to a stored list of reference features which 
30 obivates the need for pre-registering the inspected article, 
mechanically, optically, or computationally, without 
requiring extra hardware and with little extra computational 
load, and which in addition compensates for various kinds of 
scanner geometrical distortions and production errors of 
35 scaling in the article. 
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According to the present invention ^ ther^e is 
provided a method of inspecting an article with respect to a 
stored list of reference features each identified by its 
coordinates, comprising: scanning the article to generate 

5 online data corresponding to the features, called online 

features, and their coordinates detected during the scanning 
of the article; transforming the coordinates of the detected 
online features according to registration transformation 
parameters to correct for misregistration of the article; 

10 and comparing the transformed coordinates of the online 

features with those of the reference features for matches; 
characterized in that an initial estimate of the required 
registration transformation parameters is provided and is 
dynamically improved by continuously computing and 

15 periodically updating said estimate from the coordinates of 
the online features detected during the scanning of the 
article and the coordinates of the corresponding reference 
features . 

The "misregistration" of the article to be 

20 corrected includes any disparity between the reference 

feature locations and the scanned feature locations, whether 
the disparity is introduced by article placement, scanner 
errors, or production scaling errors. 

According to further features in the preferred 

25 embodiment of the invention described below, the original 
estimate of the required registration transformation 
parameters is provided by: scanning an initial small area 
of the article; detecting the online features and 
coordinates in the initial area; and computing the initial 

30 estimate of the required registration transformation 

parameters based on the relation of the detected online 
feature coordinates with respect to the reference feature 
coordinates. 

According to further features in the described 

35 preferred embodiment, the initial estimate of the required 
registration transformation parameters is dynamically 
improved by: continuing the scanning of further areas of 
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the article after the initial area; detecting further online 
features and their coordinates; and utilizing the relation 
of the further online features with their coordina.tes to the 
reference feature coordinates, for updating the estimate of 
5 the registration transformation parameters* 

According to still further features in the 
described preferred embodiment , computing the initial 
estimate of the required registration transformation 
parameters includes: determining, for each detected online 
10 feature in the initial area, all the reference features of 
the same type and with a maximal misregistration distance 
from the detected online feature; identifying such, online 
and reference features as matching pairs; and utilizing the 
matching pairs for computing the initial estimate of the 
15 required registration transformation parameters* 

According to still further features in the 
described preferred embodiment, the initial estimate of the 
required transformation parameters is periodically updated 
by: providing a plurality of registers for accumulating the 
20 coordinates of the online and reference features; 

incrementing the registers for the respective cooirdinates 
for each matching pair; and utilizing the current values of 
the registers for updating the transformation parameters* 

As will be described more particularly below, such 
25 a method, in which the coordinates of the detected features 
are transformed by dynamically changing parameters, 
eliminates the need for mechanical aids, such as 
registration pins, and also the need for an extra scan of 
the article when the registration is effected by fiducial 
30 marks . A particularly important advantage of the novel 
method of the present invention is that it reguixes very 
little extra, in terms of time as well as resources, over 
the standard method of feature matching as in the prior art. 
The method of the present invention also requires 
35 substantially simpler computations, as compared to the prior 
systems based on pixel-wise correlating reference image to 
online image, and can be implemented by software included in 



wo 93/15474 



PCT/US93/00791 



general puirpose computers , 

The inspection system is particularly advantageous 

in the otherwise difficult situations where the actual 

misregistration distance is larger than the distance between 
5 features, or where the features have a repeating pattern of 

a period smaller than the misregistration distance. 

Another important advantage is that the system 

does not rely on the existence of fiducials or other 

landmarks in the article* 
IQ The misregistration data outputted by the novel 

system may also be utilized for finding defects in various 

other types of inspection systems where good registration is 

needed. 

Further features and advantages of the invention 
15 will be apparent from the description below. 

BRIEF DESCRIPTION OF THE DRAWINGS 
The invention is herein described, by way of 
example only, with reference to the accompanying drawings, 
wherein : 

20 Fig. 1 is a block diagram illustrating an overall 

automatic optical inspection system constructed in 
accordance with the present invention for optically 
inspecting PCBs; 

Fig. 2 illustrates an inspected PCB with some 
25 features marked on it; 

Fig. 3 graphically illustrates inspected board 
features of Fig. 2 as imperfectly registered with respect to 
corresponding reference features; 

Figs. 4, 4a and 4b illustrate the manner of 
30 scanning a PCB when optically inspecting it; 

Fig. 5 illustrates a list of features arranged 
according to a lexicographical ordering; 

Fig. 6 is a block diagram illustrating the 
hardware in the comparator unit in the optical inspection 
35 system of Fig. 1 ; 



wo 93/15474 



- 7 - 



PCT/US93/O0791 



Fig* 7 is a flow diagram illustrating the overall 
inspection method; 

Fig. 8 is a flow chart illustrating the bootstrap 
operation for producing the initial and updated estimates of 
5 the transformation parameters; 

Figs, 9-12 are flow charts illustrating in moire 
detail certain steps in the bootstrap operation shown in the 
flow chart of Fig. 8; 

Fig. 13 is a flow chart illustrating the overall 
10 inspection process; and 

Fig. 14 is a flow chart illustrating an optional 
operation when periodic patterns are revealed in the 
inspected PCB. 

DESCRIPTION OF A PREFERRED EMBODIMENT 
15 Overall System 

Fig. 1 is a block diagram illustrating the various 
units of an automatic optical inspection system for 
optically inspecting printed circuit boards* 

Thus, the PCB1 is first scanned by a scanner unit 
20 2, which outputs 8-bit gray level data which, after being 

converted by binarizator 3 to binary data (black, white), is 
.fed to a feature detector unit 4. Unit 4 detects the 
various features of the inspected PCB and identifies each 
feature by its type (e.g., junctions, pads, open ends, 
25 pinholes, islands, etc.) and by its coordinates. This data 
is stored in the form of an online features list 5. 

The online features list from memory unit 5, and 
the reference features list previously stored in memory 6, 
are inputted into a comparator 7 which compares the two 
30 lists of features for excess features and missing features 
in the light of prespecified parameters defining the 
misregistration bounds inputted into the comparator 7 via an 
input unit 8, The prespecified input parameters include the 
values of Q, R, 9^^^ a^^, P^^^, P^^^ and where: Q is 
35 the reference tolerance; R is the maximal assumed 

misregistration radius, which is an upper bound on the 
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distance between any point in the scanned area of the PCB 

and its position in the reference coordinate system; ^^^^ is 

a maximal assximed misregistration angle between the 

inspected PCB and the reference; a is the maximal assumed 

max 

5 angular deviation between the light sensor array and the 

Y-motion; and P^ are the maximal assumed scaling 

Xmax xmax 

errors; and is the position error • These misregistration 
bounds depend on the quality of the preliminary alignment 
between the loaded PCB and the reference^ on the scanner 

10 precisiony^ and on the precision of the production and 
measuring tools . 

Comparator 7 outputs to a memory 9 a list of 
excess features^ namely features present in the online list 
but not in the reference list, and also missing features, 

15 namely features in the reference list but not in the online 
list. 

Fig. 2 illustrates the image of a PCB being 
inspected having two types of features: pads (P) and 
junctions (J), with a defect D in the form of an open end 

20 (OE) at the central pad (P). Fig, 3 is a graphical 

illustration of the two features lists: the upper case 
letters "P" and "J" are the reference features, and the 
lower case letters are the online features of the inspected 
PCB. It will be seen that the online feature list also 

25 shows the defect OE (open end) which is seen as an excess 
feature present in the online but not in the reference. 

The scanning of the PCB is performed in a 
plurality of slices, as shown in Fig. 4, with each slice 
partially overlapping the preceding slice. The overlap is 

30 provided to assure that a complete feature is detected. Each 
slice is scanned by moving the table TB (Figs. 4a, 4b) on 
which the article AR is mounted in the Y-direction. A 
linear array of light sensors SA is positioned 
perpendicularly to the direction of motion. Between the 
35 slices, the optical apparatus is moved in the X-direction 
(to the right). The first few features are localized in the 
beginning of the first slice. Consecutive features may be 
expected to lie relatively near each other, at least in 
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their Y-coordinates , as the temporal arrival order- of the 
features parallels their geometric order. Preliminary 
misalignment may be in the order of 0. 5-2,0 mm, and could 
even be much larger. 

5 As an example, a segment o£ a feature Ixst is 

illustrated in Fig. 5. It can be seen that the list begins 
with the first Y-coordinate 2000, which corresponds to two 
features with X-coordinates 12634 and 12644. The types of 
these features (marked by T) are identical. This means that 

10 two features of the same type are located on the same line- 
As mentioned above, the comparison in comparator 7 
(Fig. 1) compares both types and geometric positions {x,y 
coordinates)- Many algorithms are known in the prior art 
for comparing feature types according to the particular 

15 application, and therefore no specific type-matching 
algorithm is described herein. 

Fig. 6 schematically illustrates the inspection 
apparatus as including a master processor 10 for controlling 
the scanning operation, including image processing, feature 

20 detection and feature comparison as illustrated in Fig. 1 . 
The system further includes another processor 1 1 which acts 
as a slave to carry out the tasks of comparing the reference 
feature lists and the online feature lists, and of 
dynamically registering them. The slave processor 11 

25 includes a memory area 1 2 which is programmed according to 
the software required for comparing the reference and online 
feature lists and dynamically registering them. The memory 
area 12 of the slave processor 11 includes memory area 5 
(Fig, 1) for storing the reference feature list, memory area 

30 6 for storing the online feature list, and extra memory 
area, indicated at 13, for storing the program, buffers, 
registers, etc., to enable the system to carry out the task 
of comparing the reference and online features lists and 
dynamically registering them. The size of the memory 1 2 in 

35 the slave processor 11 is therefore proportional to the 
expected number of features in a single PCB. As one 
example, the slave processor 11 may be an Intel 8 0386 
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digital processor having a memory of 1 Mb, enabling it to 

handle approximately 50 K features. 

As described earlier, at some time before the 

inspection a reference list is created from a flawless 
5 sample according to the type of PCB to be inspected. Prior 

to the initiation of an inspection scan, the memory area 12 

in the slave processor 11 (which serves the comparator unit 

7 in Fig- 1), is loaded with the necessary software, the 

reference list, and also the input parameters (Q, R, ^^^y. 
10 and |i, Fig. 1) which define the misregistration bounds and 

which depend on the quality of the preliminary alignment 

between the loaded PCB and the reference. 

The system may then be used for scanning the PCB 

to generate the online feature list, which is compared with 
15 the stored reference feature list to locate defects (excess 

features or missing features), as indicated by box 9 in 

Fig, 1 , 

As will be described more particularly below, the 
method involves two stages: (1) a preliminary registration 
20 stage, in which a "bootstrap procedure" is performed to 

compute a first estimate of the relative coincidence of the 
two lists and the transformation to be applied on the online 
list bringing it to the reference coordinate system; and (2) 
a main matching stage, involving two tasks ("processes"), 

25 namely: (a) comparing the two lists after introducing the 
initial estimate of the transformation parameters; and (b) 
dynamically improving the transformation parameters as the 
online data arrives from the continuation of tliB scanning of 
the PCB after the initial area, 

30 Fig. 7 more particularly illustrates the 

comparator flow chart. Thus, the system first scans an 
initial area of the PCB until "N" (e.g., 128) online 
features have been detected, or the end of a slice has been 
reached (box 20), If in scanning the initial area the 

35 predetermined number (N) of online features has not been 
detected (box 21), the inspection process is terminated 
since there is insufficient data (box 22). 
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If N online features have been detected, the first 
stage, namely the bootstrap procedure stage is called (box 
23). In this stage, an initial estimate of the registration 
transformation parameters is computed based on the relation 
5 of the detected online feature coordinates with respect to 
the reference feature coordinates. The bootstrap procedure 
indicated by block 23 in Fig, 7 is more particularly 
illustrated in Figs. 8-12 to be described below. 

After the initial estimate of registration 
10 transformation parameters has been computed, the system 

continues to scan further areas of the PCB to detect further 
online features and their coordinates, and utilizes their 
relation with the reference feature coordinates for updating 
the estimate of the registration transformation parameters. 
15 The updated parameters are then utilized in a fine- 
registration procedure (more particularly shown in Fig. 13) 
to transform the coordinates of the further online features 
(box 24). The system finally outputs a list of the defects, 
i.e., excess features and missing features (box 25) • 
20 To carry out the above computations, a 

mathematical model of the misregistration transformation is 
built. This allows devising an efficient algorithm for 
. computing the model parameters, given the inspection data, 
and also controlling corection transformations introduced 
25 into the system (which are restricted to the allowed model), 
thus preventing defect misdetection. 

The misregistration transformation model is 
characterized by the article production and mounting errors 
and by the scanner repeatable geometrical distortion which 
30 affect the geometrical system produced by the scanner, 
including the following kinds of flaws /misalignments : 

(a) the angular deviation (a) caused by the light- 
sensor array not being exactly perpendicular to the Y- 
direction motion; 
35 (b) misregistration angle (9) caused by the 

article not being exactly parallel to the reference because 
of the mounting misregistration, and/or the angular 
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discrepancy between the table and motion direction (scanner 
distortion); 

(c) scaling errors in both directions (P and P ) 
because of the article production and/or scanner problems; 

5 

(d) optical distortions along the sensor axis, 
especially variations in magnification (because of varying 
distance h (Fig. 4b) between the sensors and the article 
surface) . 

10 The Bootstrap Procedure 

Fig. 8 is a flow chart illustrating the bootstrap 
procedure represented by box 23 in Fig, 7, and Figs. 9-12 
are flow charts illustrating more particularly some of the 
operations involved in the bootstrap procedure of Fig. 8- 

15 It can be shown that for a single slice said 

distortion effects (except optical distortion of a scanner) 
can be modeled as an affine transformation utilizing six 
vairiables : 

a, b, c, d^ 5x and 6y : 
20 = ax + by + 5x ^ Kg, 1 

y^ = cx + dy + 5y 

where 6x, 6y are translational part of misregistation 
correction transformation parameters; a,, b^ o, d are 
multiplicative part of misregistration correction 

25 transformation parameters; x, y are the coordinates before 
registration / and and y'^ are the coordinates after* 
registration (i.e., in the reference space). 

The function of the bootstrap procedure is to 
produce a first estimate of said misregistration corx-ection 

30 transformation parameters, which will be continuously or 
periodically updated once the bootstrap procedure is 
completed and as more online data becomes available. 

In case where only article placement correction is 
required, the bootstrap procedure computes trans formaLt ion 

35 parameters of the special, simpler form as follows: 
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a = cos 9, b = -sin 0, c = sin G, d = Cos 0 Eg , 1 a 

In the more general case, connection between the model 

parameters a, b, c, d and the physical scanner distortion 

and article placement parameters 0, a, S^, can be shown 

5 as follows : 

a = S cos e/cos a b = -S sin e/cos a 

^ Eg, lb 

c = Sy sin (e-a)/cos a d = cos {e-a)/cos a 

Different slices can be shown to differ onlyin their 6 and 

5 but all have the same a, b, c and d parameters, 
y 

10 As shown by the flow chart in Fig. 8, the first 

step in the bootstrap procedure is to compute a displacement 
vectors list for each online feature (box The flow 

chart of Fig. 9 more particularly shows how this is done. 

Thus, as shown in the flow chart of Fig. 9, for 

15 each of the N online features (box 31), and for all the 

reference features within the misregistration radius R (box 
32), detemine whether a type match exists (box 33). 

Next, in order to determine whether a match exists 
between an online feature with coordinates i^^f Y^) ar^d type 

20 T^, and a reference feature with coordinates (x*,y') and 
type T*, the following must hold: 

Ix'^x^Ur and ly'-y^^UR Ea> 2 

and type T^ must match the type T' . The pair, of online 
feature with its matching reference feature, is called a 

25 matching pair. 

Type matching procedures are well known in the 
prior art according to the particular application. The 
decision rules are usually derived heuristically , sometimes 
by the user. Since the novel system is not concerned with 

30 any particular type maching procedure, a particular type 
matching procedure is not described for the sake of 
conciseness . 

The next step (box 34, Fig. 9) is to compute and 
record, for each maching pair found, the displacement 
35 vector, according to the following equation: 
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(A^, ^ y> = y'-^n^ 

The so-computed displacement vectors are used in 
producing a displacement vectors list of the N online 
features (box 35, Fig, 9). 

5 The full reference list is ordered in increasing 

lexicographical order, primarily by y and secondarily by x, 
and therefore the n'th displacement- vectors list is 
similarly ordered by virture of being a sublist of the full 
reference list from which a constant vector (x^, y^) has 

10 been subtracted. This greatly facilitates the task of 

producing one sorted list from the N displacement-vectors 
lists, by successively merge-sorting them (cf . The Art of 
Computer Programming D.H, Knuth, Reading, Mass. Addison- 
Wesley, 1968., vol, 1, section 2.3.3). Displacement vectors 

15 appearing in more than one list are found during the merges ,^ 
and recorded only once, with a population count. The 
displacement-vectors lists of the online features are thus 
merged into one list of displacement vectors (box 35). 

The next operation performed by the bootstrap 

20 procedure illustrated in the flow chart of Fig* 8 is to 

estimate the correction parameter 5 , 5 , as shown by block 

X y 

40 in Fig. 8. This requires a correlation function, W(), to 

• be computed. The flow chart of Fig. 10 more particularly 

illustrates how this is done. 

25 Thus, as indicated by box 41 in Fig. 10, the size 

of the scan zone of the first N online features is 

determined by computing the bounding-box of these features . 

The size of the bounding box is denoted by b x b ; that 

X y 

is : 

b =max X -min x , b =max y -min y_ Eg. 4 

30 x n n' y -^n -'n 

The amount of "allowable noise" in the correct 
displacement vector (the one best describing the 
misregistration in the initial scan zone) is then determined 
(box 42), This amount is defined separately in the two 
35 directions (x,y) and is termed the "uncertainty-box". 

Contributing to its size are two factors: The first factor 
(and usually the larger) is the angle (9) between the PCB 
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and the reference, which appears as '*noise" in this phase 

when using only displacement and not the angle* This factor 

is estimated by the size of the bounding-box and an upper 

bound on the angle (input parameter 6__„) • 

majc 

5 The second factor encompasses all other noises in 

the reporting of the position of a feature, including image 

quantization (pixel size), feature-detection mechanism 

errors, scanner mechanics, scanner unmodeled geometrical 

distortion, etc. An upper bound on this factor is teirmed 

10 "position error" and is denoted by the input parameter 'Vq*** 

Finally, the uncertainty-box size, denoted by M x M , is 

X y 



computed as follows : 

M =b e +vi 
X y max 



M =b 8 +vi , M =b e +y. Eg. 5 
X y max '^x' y x max '^y — ^ 



15 u=u +a b+(P^ )'b 

^y '^oy max x ^ Ymax y 



Eg. 5a 



Where P„ ^„ (P^ is a maximal assumed difference between 

Xmax Ymax 

P ( P ) and 1 • 
X y 

For each vector u=(u ,u ) in the displacement- 

X y 

vectors list, we compute W(u), the correlation function 
20 applicable to our situation. The correlation function W(u) 
is the number of displacement-vectors v=(v ,v ) whichi lie in 

^ y 

the uncertainty-box centered around u, that is, whicli 
satisfy: 

|v -u |<M /2, and |v -u I <M /2, Eg, 6 

' X X* X ' y y y 

25 with population taken into account. This computation is 

relatively intensive, but the fact that the displacement- 
vectors list is sorted provides an efficient algorithm for 
computing the function W(u) (box 44). 

The main loop is on u (a vector in the 
30 displacement- vectors list) in the order of the displacement- 
vectors list* Thus, successive u vectors come in increasing 
lexicographical order. For each u we shall compute W(u) 
(box 44), The manner of doing this is more particularly 
illustrated in the flow chart of Fig, 11. 
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Thus, as shown by box 45 in Fig. 11, we first 
define two pointers, Y^^^ and Y^igh' displacement- 
vectors list. Throughout the u-loop, Y^q^ points to an 
element of the displacment-vectors list whose y value is the 

5 smallest in the list which satisfies y > Uy-My/2; and Y^igh 
points to an element of the list whose y value is the 
smallest in the list, which satisfies y > Uy+My/2. These 
pointers need updating only when u^ increases, that is, when 
proceeding to the next u in the u-loop (box 46) and 

10 changes . 

The update is a simple matter of incrementing each 
pointer until it satisfies its "normal" condition again. u 
defines a set of vectors in the displacement -vectors list 
whose y value satisfies u^-M^/Z <y < We shall term 

15 this set the y-window (in a sense it is a "window" on the 
full displacement-vectors list, defined by limiting 
y-values). Computationally, this set is simply all the 
displacement vectors whose address (in the displacement- 
vectors list) satisfies Y^q„ ^ address < Yj^^gh* 

20 We shall use the term "y-run" for the set of all 

displacement-vectors with the y- value. These displacement 
vectors are arranged consecutively in the displacement- 
vectors list and are ordered by increasing x-values. 
For each distinct y in the y-window, we keep a pair of 

25 pointers, X{y)j^^^ and XCy)^^^^^. During a u-loop phase when 
u stays constant, these pointer-pairs maintain the 
following conditions, similar to Yj^^^ a^nd Yj^^^j^: ^^^^low 
points to an element in the y-run whose x value is the 

A 

smallest ajid which satisfies x > and ^tyij^igh 

30 points to an element in the y-run whose x value is the 

smallest and which satisfies x > u +M /2, Whenever u 

increases, we update and Yj^^gj^ (box 46), and we reset 

all these pointer-pairs by .pointing each pair to the 

beginning of its run (box 49). In each subsequent pass 

35 through the u-loop, when u increases and u does not 

X y ^ 

change, all these pairs are incremented appropriately (in 

A A 

the same manner as ^-^ and (box 47). 
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10 



15 



20 



25 



30 



Now, all displacement vectors v contributing to 
W(u), which were defined to be those satisfying 
|v^-u^| iM^/2, and ] v^-u^ 1 sMy/2 , are the union of sets 
corresponding to all y's in the y-window. Each such set 
consists of all displacement vectors whose address satisfies 
X(y)^Q^< address <X(y)j^igh. That is, W(u) is determined by 
counting all the displacement vectors (v) in^the uncertainty 
box window using the Y^^^^ and \. and the X(y)iQ„ and the 
X(y)^^ pointers (box 48). 

Once W(u) is computed (box 44, Fig. 10), we find 
the best u, i.e., maximizing W(u) (box 50, Fig. 10). For 
this purpose, a vector v is found, for which W(v)>W(u) for 
all u's. Then we determine (box 51) the vector w that 
maximizes W(t) among all vectors t=(t^,ty) which lie outside 
the uncertainty-box centered around v, that is, which 
satisfy 

IV^xl'V2°^ IV^yl'V^ 

It is helpful to think of v as the "most popular 
displacement vector", and of w as the "second most popular- 
displacement vector" . If 

W(v)-W(w) 

< 0.1 

W(v)+W(w) 

we deduce that the second most popular differs very little 
from the most, popular vector, and so we suspect that the 
board is periodic (box 52) or has some other difficult 
structure; in such case we may revert to a special case 
algorithm (box 53) for computing w for periodic patterns. 
This special case is described below with reference to the 
flow chart illustrated in Fig. 14. 

We can now estimate the translational part 
of the registration transformation parameters (box 54, Fig. 
10). For this purpose we compute the 2-dimensional 
gaussian-weighted average, 

5=(5x,6y), of all vectors t which satisfy 
It -V |<M 2 and jt -v^l<M^/2. ^3^-3. 

' x X ' X ' y y ' y 
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Returning to the boostrap flow chart of Fig. 8, 
after the translational parts (5^, 5„) have been estimated 

(box 40, Fig. 8), the multiplicative parts (a, cv d) of 
the registration transformation parameters are now 

5 determined, and the translational parts are re-estimated 
(box 60), thus completing the bootstrap estimation of the 
misregistration parameters. To this end, we match eacti of 
the N online features (whose coordinates are indicated as 
x.,y^) against each reference feature with matching type and 

10 with coordinates (x',y')/ such that 

lx»-(x,+5 )l<M^/2 and | y ' - (yi . 5^) i >M /2 , Eq> 9 

1 X X * J X 

Note that an online feature may have more than one matching 
reference feature . 

The memory area 13 (Fig. 6) of the slave processor 
15 11 includes eleven accumulating registers, named: 

S, S^, Sy, S^., Sy,, S^^., Syy,, S^y,, Sy^. 
These registers are used in re-estimating the registration 
transformation parameters, as shown by the flow chart in 
Fig. 12. 

20 Thus, according to the flow chart of Fig* 12 the 

eleven registers are reset (box 61 ) . Each of the N online 
features (box 62) is paired with each of the matching 
reference features according to Equation 9 above, and. the 
eleven registers are appropriately incremented (box 64). 

25 Each of the above registers thus holds a sum to 

which a value is added for each matching pair. The 
registers are accumulated as follows: (x,y) and (x*,y') 
denote a specific matching pair; the sums range over all 
matching pairs encountered up so far: 

30 



35 



s 




El 






Ex 


s 

y 




Ey 






Ex' 


\' 




Ey' 






Exx' 


S , 

yy 




Eyy' 


s , 

xy 




Exy ' 



Eg. 10 
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S , = Eyx' 
yx' 1L 

S = 2x 

s = 2 y 
yy 

A better approximation of the registration transformation 
5 parameters (for correcting article placement only) can now 
be computed (box 65) using the current values of the eleven 
registers, according to the following equations: ^ 

S S , 

Wxx' = Sxx* Ea^11 



s s , 

Wyy* - Syy' ^— ^ Ea>12 



10 



15 



S S . 

Wxy' = Sxy* ^ ^ Ea.1 3 



S S , 

Wyx' = Syx' ■ Ea>1 4 

S 

S 2 

W = S — Eq.15 

XX XX „ 

^2 
S 2 

W = S ^ Eg.l 6 

yy yy c 



V = V(W ,+W .)^+(W ,-W Eg . 1 7a 

XX yy yx xy _— — 

(W ,+W ,) 

COS e = — ^ — Eo.17b 



(W„„,-W , ) 

sin e = — ^ — Eg. 17c 

V 

S , - (S cos 8 - S sin 9) 

6 = ^ ^ Eq . 1 Id 

^ S 

S I - (+S sin 6 + S„ cos 6) 
5 = — ^ Ea.17e 
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According to Eg. la: a - cos 6; b = -sin G; c = sin 0-; and 
d = cos Q 

In the more general case, an estimation for the 

full af fine transformation canbe computed accordingto the 

following equations: 

V = W W -W^ 
XX YY xy 



a = 



b = 



c == 



w w - w w 

yy XX xy \ 







V 






w 

XX 


yx' 




w 

xy 


W , 

XX 






V 






W 

YY 


W , 

xy 




W 

xy 


W , 

yy 






V 






W 

XX 


w , 

yy 




W 

xy 


xy' 



Fior. 


18a 


Fia* 


18b 


Ficr. 


18c 


FicT. 


18d 


Fia. 


18e 



10 d = 



The above mathematical method of computing the 
registration transformation is based on the theory of 
least-squares data fitting, described, for example, in W.H. 
Press, B*P* Flannery, S-A. Teukolsky and V7.T. Vetterling, 
15 Numerical Recipes, Cambridge University Press, 1986, 

Our equations arise by applying the theory to the 
following problem: Given (x^,y^) and (x'^,y'^), i=1..-n, 
find 6x,6y and a, b, c, d which minimizes the following 
expression: 



20 /_lllcd| yy-}'' K6ZrVY:'l\\ . laU9 

For article placement only, the minimization is 
constrained to rigid transformations. 
The Main Procedure 

Following the completion of the bootstrap 
25 procedure (box 23, Fig. 7) for determining the registration 
transformation parameters on the initial scanned area, the 
system operates according to the main procedure. This 
consists of two processes working simultaneously, namely a 
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comparison process and a fine-registration process, as 
indicated by box 24 in Fig. 7. The main procedure is more 
particularly illustrated in the flow chart of Fig. 13. 

Thus, as indicated by box 70 in Fig. 13, when an 
5 online feature appears, with coordinates (x.y), its position 
is transformed, using the available transformation 
parameters, so as to be registered with the reference 

features, giving new coordinates (x^,y^) using Equation 1. 

r r • • 

Now, using these coordinates (x ,y ), the reference list is 

10 searched for every feature with coordinates (x*,y') that 

satisfy : 

jx'-x^l 5 Q and ly'-y^l ^ Q , Eg^_20 
where Q is the reference tolerance, and of a type matching 
the online features type (box 71).' 

15 As indicated earlier, the transformation found in 

the booststrap procedure aligns the online features with the 
reference features in the initial scan zone. The boots tirap 
procedure operates on only a few online features, so that no 
time is lost waiting for a large number of online features 

20 to accumulate. The transformation produced by the bootstrap 
procedure may therefore not be the best transformation for 
the complete scanning zone. Fortunately, in the initial 
• scan zone, better alignment is not yet required. However, 
as the scan continues over the complete scanning zone, a 

25 fine-registration process operates together with the main 
matching process in order to improve the transformation 
parameters . 

Thus , as shown in the flow chart of Fig • 13/ as 
each online feature is detected during the inspection 

30 process, it is compared with the stored reference features 
to see whether there are any matching pairs (box 72). If an 
online feature does not find a match with a stored reference 
feature, the online feature is recorded as an excess feature 
(box 73). On the other hand, if an online feature is 

35 matched with a stored reference feature (box 72), the eleven 
registers within the memory area 13 are updated (box 75) as 
described above, and the matching reference features are 
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marked as "not missing" (box 76). This updating requires 
very little computation - only eleven additions and six 
multiplications . 

Whenever it may be decided to update the 
5 transformation parameters (box 77)^ e.g., when enough online 
features have been collected or the scan has proceeded a 
large distance since the last update, this may be done (box 
78) according to the current state of the eleven registers 
as in Equations 1 0-1 8 . 

At the end of the scan (box 74), all the reference 
features not marked as "not missing", are reported as 
"missing" (box 79). The system now reports a list of the 
excess features and missing features (box 25, Fig. 7). 

The transformation parameters are also reported in 
15 order to facilitate high-level control or recording of the 
scanner geometrical distortion parameters derived from the 
transformation parameters. 

It will thus be seen that, after the initial 
bootstrap procedure, the scan proceeds to pirovide more 
20 online features detected in the inspected PCB. This data 
may be used for updating the initial transformation 
parameters so as to increase their precision with the 
newly-arriving data. Athough the transformation parameters 
applied to the initially-scanned area are imprecise, they 
25 generally would be sufficient for the description of the 
transformation at the locale of the initially-scanned area 
so that it would not be necessary to return and rescan the 
initially-scanned area. Indeed, in principle the angle G", 
for example, cannot be measured accurately by using only a 
30 small zone. On the other hand, the exact angle is not 

necessary for accurate comparison of the initial zone to the 
reference. Only as the scan proceeds, a better approximation 
of the angle is needed, and is obtained with the arrival of 
more online data. Thus, the whole combined process of 
35 inspection and registration may be performed in a single 
scan. 
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When the scan of the first slice is finished, a 
good approximation of the first-slice transformation is 
achieved. However, as mentioned before, if scannner 
compensation is required, this transformation is only 

5 partially covered for the second slice. The parameters a, 
b, c, d are correct, but 5^ and 5^ for the second slice may 
differ slightly from those of the first. Thus, when scanner 
compensation is requested, the processing of the second- 
slice features is also preceded by a boot-strapping phase 

10 which uses the transformation from the previous slice. This 
bootstrap procedure operates with a much smaller search 
radius (R) , assumes no angular discrepancy from the previous 
transformation (0^^^ = 0, a^^^ = 0, P^^^^ - 0, P^^^^ - 0), 
and restricts itself to finding only translational 

15 deviations, thus changing only 5^ and 5^. 
Optional Algorithm for Periodic Patterns 

As mentioned above with respect to box 53 in the 
flow chart of Fig, 10, if a periodic board is found (box 
52), a special algorithm for periodic patterns should be 

20 applied. 

In the periodic case, the correlation function 
W() will have more than one significant peak. These peaks 
■ form geometrically a 2-dimensional grid. The first phase 
analyzes the correlation function W() to find the shortest 
25 grid generators. Generators of a 2-dimensional grid G are 
two linearly- independent vectors g^ and ^2 that all 
vectors in G can be obtained as a linear combination of g-j 
and ^2 with coefficients hg^ "^^^2 ^ belong to 

-2, -1, 0, 1, 2, 3, ] (there may be only one 

30 generator if the periodicity is 1 -dimensional) , This is 

done by first applying a threshold to W() with the value of 
80% of the highest value in W(). Then the remaining 
displacement vectors are clusterized to form a 2-dimensional 

set of points P-^ P^^ located at the cluster centers. Next, 

35 for each point in the set, , the following is done: 

1. The point in the set P^....Pj^ which is closest 
to P- and is not P- itself is found. Call it q-^ * 
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2. The subset of points that are not almost- 
collinear with ^^-<^^ is formed and from it the point closest . 
to is found. We define "almost-collinear" as a third 
point V that makes the angle between V-P^ and V-q^ — between 

5 +15° and -15° or between ia0° + 15° and 180°-15°. Call it g^- 
The above subset may be empty, in which case ^2 
considered to be not found. 

3. ^i^^T ^1~*^1 entered into a vector list 
called the "generator vector list". If g2 was found <^2''^^ 

10 and P^-q2 B.re entered into the list. 

After going over all the points (P^ ) and doing the 
above, the generator vector list is complete. 

Now we take from this list the most abundant 
vector and name it g^ . Next, we search the list for the 

15 most abundant vector among those that are not almost in the 
direction of g^ (again, in the 15*" criterion)- If there is 
no such vector, g^ is the only generating vector. Otherwise, 
if we name it g^ and g-j , g2 are the two generating 
vcectors , 

20 Now that we have found the generators ( s ) we apply 

this knowledge to the feature-sets (online and reference) to 
find period-breaking features. In each of the two lists we 
do the following: for each feature having position f , we 
search in its own list for features in the proximity of 

25 positions f+g^^ and f -g^ , having a type that matches f 's. If 
one (or both) of these positions has no matching features ^ 
and is inside the inspection zone (or slice boundaries, for 
the online list), f is marked as a period-breaker of type 1. 
The same is done for g^ (searching positions f+g2 ^i^^ ^"^2 

30 and marking as type 2), ±t indeed exists. We continue to 
process the list until enough period-breakers of type 1 are 
found and if ^2 ^^i^ts, enough period-breaker ts of type 2 
,are also found. 

We define the following vectors: 

25 if g2 exists, n _ i » _ i 
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"2 



otherwise, U 



1 



"2 





And U 



II °2ll 



The vector will serve as the direction in which a 

1 -dimensional correlation wil be computed between the online 

and reference period-breakers of type 1 : 

For each online feature which is a period-breaker 

10 of type 1 having coordinates (x, y) we search for all 

reference features within distance of R from (x, y ) , which 

are also period-breakers of type 1 and which have a type 

matching that of the online features. For each reference 

1 1 

feature having coordinates (x , y ) we compute: 
15 z = (x'* - + (*y'' - y)Uiy 

and increment the count for Z in the correlation fuction 

{ ) , i.e., increment C^(2), Z being the range -V2R. . .V2R 
(therefore is a vector in the range -V2R...V2R). If g^ 
exists we construct in the same way (looking for type 2 

20 period breakers and using and ^2^) . 

If g^ does not exist we construct C^^i) with type 1 
period breakers, still using U2^ and U^y- 

Now we find so that C^(z^) is maximal in ( ) , 
and Z2 so that 02(22) is maximal in C2(). From bhese 
25 compute the displacement vector V: 

V = Z^U^ + Z- 

X 1 1x 2 2x 

V = Z.U- + 2^ U« 

y 1 iy 2 2y 

This V vector is the estimation of the correct displacement 
vector amongst the several that had high score in the 
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original correlation function W( ) . V7e continue with V as 
in the non-periodic case (see Eq. 8 and on and box 54 in 
Fig. 10). 

Optional Registration Algorithm for Scanner Sensor Axis 
5 As mentioned before, the scanner introduces 

distortions along the sensor axis. For a given slice and Y 
coordinates^ this distortion is modeled along the scanner X 
axis - sensor axis as a polynomial of third degree. As the 
stage moves along the scanner Y axis this polynomial may 
10 change slowly. A third degree polynomial model is justified 
if we assume the distance between the light sensors and the 
article is a second degree polynomial as a function of X. 
Also lens magnification distortion as a function of X can be 
described as a second-degree polynomial. In a particular 
15 slice, the total of registration transformations becomes; 



l^x ^+P^(y) + P^(y)x + P2(y)x^+P3(y)x^^ " (y^) 



ax^ +by+5x 
1 

cx +dy+5y 




r r 

where, x, y are scanned feature coordinates, x , y 
reference (registered) coordinates. The algorithm for 

20 obtaining these extra model parameters PgCy), P1(y)f ^2^^^' 
p^Cy) will be described. 

Let us assume for the moment that a, b, c, d, and 
5 , 6 are already known . Denote by a\ b^ , c\ d , 5^ 
5y the parameters representing the inverse affxne 

25 transformation (reference to online): 

a = b = c d = 



r 



ad-bc ad-bc ad-bc ad-bc 

If (xi, yi) < > (ui, vi) are the reference- 
online matches, we wish to model the error remaining after 
said transformation (a^ui+bVi+5^x)-xi as a third-degree 
30 polynomial in xi. We do this by least-squares fitting so we 
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need the following: 

S^'^ = E[X1^], (0sk<6) 

E[xi'^(aiUi+b^vi+6x^ )1 = (OsksB) 

= a^EtXi'^Uil+blElXi'^Vi] + 6 ^Exl^ = 

A X fa 

And then we solve the 4 by 4 linear system: 
Ep = F for unknowns P =■ 

^2 
P3 

= ^^^x*^" + ^^'^x'^^ + ^x^^x*^' ^^^^^ 
m+1 



where 



10 ^ml = O^-n^S 

0il53 

This is the procedure for obtaining the third 
degree polynomial coefficients P^, P^ , P^, P3 given the 
match data (Xi Yi Ui Vi) and the affine transformation (a, 
b, c, d 6^, 6y), 

15 As can be seen, the information extracted from the 

matching data is independent of the affine transformation 
and can be gathered in fifteen accumulation registers: 
In) n k ^ 

- ^1 0=<k<«6 

(n) n k , 
SxKu - ZxjU| Oa<k<=3 

<n| n k 

20 SxKv - 2>c,V| 0-<k<=3 

where n is the number of the current match in this slice. 

It was mentioned before that the polynomial may 

change slowly with y during the slice scan. We incorporate 

this by making the registers accumulate match data with 

25 exponentially-decreasing weight for matches further back in 
Y: (n) n (yn-yi) k 

SxK « Eq X| 0='<k<«e 
l» 1 

in) n (vn-yl) k 
SxKu - Zq X|U| o=<k<=3 
1-1 

(n) n lyn-yl) k 
SxKv - X|V| o«=i<.k<=3 

1-1 
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where OLgLI , q characterizes the special frequency of the 
changes in the distance between the sensors and this 
article. 

This can also be accomplished in accumulators by: 

(n) <vn-viv-i| <n-1) k 

5 SxK » q SxK . x„ 0«<k<=6 

(nj (yn-yn-l) (n-l| k 

Sx»^u - q SxKy + X^Un 0«<k<=:=3 

(n) (yn-yn.!) (n-1J k 

SxKv - q SxKy + X„v„ 0«<l«-3 

All this match-data accumulator-gathering process 
for the polynomial is merged in with the matching and 

10 gathering formulas for the affine transformation at the end 
of the bootstrap and during the scan. During the scan, as 
each match is found, it is used to update the eleven 
"affine" registers and the fifteen "polynomial" registers. 
Periodically, the affine transformation is 

15 re-estimated using "affine" registers and the polynomial is 
re-estimated using the "polynomial" registers and the 
current affine transformation by solving the above 4x4 
linear system. 

While the invention has been described with 

20 respect to one preferred embodiment relating to compensation 
for rigid motion only, it will be appreciated that the novel 
method could also be used for compensating for any other 
type of misalignment, including parallel shift of the X-axis 
or Y-axis, or both; rotation; full rigid motion; scale error 

25 along the X-axis or Y-axis or both; perpendicularity error 
between the X-axis and Y-axis; and/or full linear 
distortion, generally expressed by the following equation 
with unknown parameters: 

= ax + by + c 

30 y - dx + cy + f 

Many other variations, modifications and 
applications of the invention may be made. 



SUBSTITUTE SHEET 



wo 93/15474 



- 29 - 



PCr/US93/00791 



WHAT IS CLAIMED IS : 

1 . A method of inspecting an article with respect 
to a stored list of reference features each identified by- 
its coordinates, comprising: 

scanning the article to generate online data 
corresponding to the features, called online features, and 
their coordinates detected during the scanning of the 
article; 

transforming the coordinates of the detected 
online features according to registration transformation 
parameters to correct for misregistration of the article; 

and comparing the transformed coordinates of the 
online features with those of the reference features for 
matches; 

characterized in that an initial estimate of 
the required registration transformation parameters is 
provided and is dynamically improved by continuously 
computing and periodically updating said estimate from the 
coordinates of the online features detected during the 
scanning of the article and the coordinates of the 
corresponding reference features . 

2 . The method according to Claim 1 , wherein said 
initial estimate of the required registration transformation 
parameters is provided by: 

scanning an initial small area of the article; 

detecting the online features and coordinates in 
said initial area; 

and computing said initial estimate of the 
required registration transformation parameters based on the 
relation of the detected online feature coordinates with 
respect to the reference feature coordinates, 

3. The method according to Claim 2, wherein said 
initial estimate of the required registration transformation 
parameters is dynamically improved by: 

continuing the scanning of further areas of the 
article after said initial area; 
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detecting further online features and their 
coordinates ; 

and utilizing the relation of the further online 
features with their coordinates to the reference feature 
coordinates, for updating the estimate of the registration 
transformation parameters* 

4 * The method according to any onee of Claims 
1-3, wherein computing said initial estimate of the ireguired 
registration transformation parameters includes: 

determining, for each detected online feature in 
said initial area, all the reference features of the same 
type and with a maximal misregistration distance from the 
detected online feature; 

identifying such online and reference features as 
matching pairs; 

and utilizing said matching pairs for computing 
the initial estimate of the required registration 
transformation parameters. 

5- The method according to any one of Claims 1-4, 
wherein the initial estimate of the required transformation 
parameters is periodically updated by: 

providing a plurality of registers for 
accumulating the coordinates of the online and reference 
features ; 

incrementing the registers for the respective 
coordinates for each matching pair; 

and utilizing the current values of the registers 
for updating the transformation parameters, 

6. The method according to either of Claims 4 or 
5, wherein said matching pairs are utilized for computing 
the initial estimate of the required registration 
transformation parameters by: 

determining, for each such matching pair, the 
displacement vector of the respective pair; 

collecting all the displacement vectors for said 
initial area; 
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computing therefrom a correlation function; 

and utilizing said correlation function for 
computing the initial estimate of the registration 
transformation parameters . 

7. The method according to Claim 6, wherein 
computing said correlation function includes: 

defining a bounding box of the features in the 
initially scanned area; 

defining an uncertainty box of the initially 
scanned area to allow for noise therein; 

and counting all the vectors in the uncertainty 
box centered around the respective vector in the 
displacement vectors list for computing the correlation 
function of the respective vector in the displacement 
vectors list, 

8. The method according to any one of Claims 1-7, 
wherein the method is used to compensate for misregistration 
of the article because of article placement errors, scanner 
errors, and/or production scaling errors. 

9. The method according to any one of Claims 1-8, 
wherein said article is optically scanned and its features 
are optically detected. 

10. The method according to any one of Claims 1-9, 
wherein said article is a printed circuit .board. 

1 1 • The method according to any one of Claims 
1-10, wherein there is produced, as a result of said 
inspection, a list of excess features present in the 
inspected article but not in the stored list of reference 
features, and a list of missing features present in the 
stored list of reference features but not in the inspected 
article. 

1 2 . The method of inspecting an article with 
respect to a stored list of reference features substantially 
as described with reference to and as illustrated in the 
accompanying drawings. 
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